<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <title>Taskflow Algorithms &raquo; Data-parallel Pipeline | Taskflow QuickStart</title>
  <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600" />
  <link rel="stylesheet" href="m-dark+documentation.compiled.css" />
  <link rel="icon" href="favicon.ico" type="image/x-icon" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <meta name="theme-color" content="#22272e" />
</head>
<body>
<header><nav id="navigation">
  <div class="m-container">
    <div class="m-row">
      <span id="m-navbar-brand" class="m-col-t-8 m-col-m-none m-left-m">
        <a href="https://taskflow.github.io"><img src="taskflow_logo.png" alt="" />Taskflow</a> <span class="m-breadcrumb">|</span> <a href="index.html" class="m-thin">QuickStart</a>
      </span>
      <div class="m-col-t-4 m-hide-m m-text-right m-nopadr">
        <a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
          <path id="m-doc-search-icon-path" d="m6 0c-3.31 0-6 2.69-6 6 0 3.31 2.69 6 6 6 1.49 0 2.85-0.541 3.89-1.44-0.0164 0.338 0.147 0.759 0.5 1.15l3.22 3.79c0.552 0.614 1.45 0.665 2 0.115 0.55-0.55 0.499-1.45-0.115-2l-3.79-3.22c-0.392-0.353-0.812-0.515-1.15-0.5 0.895-1.05 1.44-2.41 1.44-3.89 0-3.31-2.69-6-6-6zm0 1.56a4.44 4.44 0 0 1 4.44 4.44 4.44 4.44 0 0 1-4.44 4.44 4.44 4.44 0 0 1-4.44-4.44 4.44 4.44 0 0 1 4.44-4.44z"/>
        </svg></a>
        <a id="m-navbar-show" href="#navigation" title="Show navigation"></a>
        <a id="m-navbar-hide" href="#" title="Hide navigation"></a>
      </div>
      <div id="m-navbar-collapse" class="m-col-t-12 m-show-m m-col-m-none m-right-m">
        <div class="m-row">
          <ol class="m-col-t-6 m-col-m-none">
            <li><a href="pages.html">Handbook</a></li>
            <li><a href="namespaces.html">Namespaces</a></li>
          </ol>
          <ol class="m-col-t-6 m-col-m-none" start="3">
            <li><a href="annotated.html">Classes</a></li>
            <li><a href="files.html">Files</a></li>
            <li class="m-show-m"><a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
              <use href="#m-doc-search-icon-path" />
            </svg></a></li>
          </ol>
        </div>
      </div>
    </div>
  </div>
</nav></header>
<main><article>
  <div class="m-container m-container-inflatable">
    <div class="m-row">
      <div class="m-col-l-10 m-push-l-1">
        <h1>
          <span class="m-breadcrumb"><a href="Algorithms.html">Taskflow Algorithms</a> &raquo;</span>
          Data-parallel Pipeline
        </h1>
        <nav class="m-block m-default">
          <h3>Contents</h3>
          <ul>
            <li><a href="#ParallelDataPipelineIncludeHeaderFile">Include the Header</a></li>
            <li><a href="#CreateADataPipelineModuleTask">Create a Data Pipeline Module Task</a></li>
            <li><a href="#UnderstandInternalDataStorage">Understand Internal Data Storage</a></li>
            <li><a href="#DataParallelPipelineLearnMore">Learn More about Taskflow Pipeline</a></li>
          </ul>
        </nav>
<p>Taskflow provides another variant, <a href="classtf_1_1DataPipeline.html" class="m-doc">tf::<wbr />DataPipeline</a>, on top of <a href="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr />Pipeline</a> (see <a href="TaskParallelPipeline.html" class="m-doc">Task-parallel Pipeline</a>) to help you implement data-parallel pipeline algorithms while leaving data management to Taskflow. We recommend you finishing reading TaskParallelPipeline first before learning <a href="classtf_1_1DataPipeline.html" class="m-doc">tf::<wbr />DataPipeline</a>.</p><section id="ParallelDataPipelineIncludeHeaderFile"><h2><a href="#ParallelDataPipelineIncludeHeaderFile">Include the Header</a></h2><p>You need to include the header file, <code>taskflow/algorithm/data_pipeline.hpp</code>, for implementing data-parallel pipeline algorithms.</p><pre class="m-code"><span class="cp">#include</span><span class="w"> </span><span class="cpf">&lt;taskflow/algorithm/data_pipeline.hpp&gt;</span></pre></section><section id="CreateADataPipelineModuleTask"><h2><a href="#CreateADataPipelineModuleTask">Create a Data Pipeline Module Task</a></h2><p>Similar to creating a task-parallel pipeline (<a href="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr />Pipeline</a>), there are three steps to create a data-parallel pipeline application:</p><ol><li>Define the pipeline structure (e.g., pipe type, pipe callable, stopping rule, line count)</li><li>Define the data storage and layout, if needed for the application</li><li>Define the pipeline taskflow graph using composition</li></ol><p>The following example creates a data-parallel pipeline that generates a total of five dataflow tokens from <code>void</code> to <code>int</code> at the first stage, from <code>int</code> to <code>std::string</code> at the second stage, from <code>std::string</code> to <code>float</code> at the third stage, and <code>float</code> to <code>void</code> at the final stage. Data storage between stages is automatically managed by <a href="classtf_1_1DataPipeline.html" class="m-doc">tf::<wbr />DataPipeline</a>.</p><pre class="m-code"><span class="cp">#include</span><span class="w"> </span><span class="cpf">&lt;taskflow/taskflow.hpp&gt;</span>
<span class="cp">#include</span><span class="w"> </span><span class="cpf">&lt;taskflow/algorithm/data_pipeline.hpp&gt;</span>

<span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>

<span class="w">  </span><span class="c1">// data flow =&gt; void -&gt; int -&gt; std::string -&gt; float -&gt; void </span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Taskflow</span><span class="w"> </span><span class="n">taskflow</span><span class="p">(</span><span class="s">&quot;pipeline&quot;</span><span class="p">);</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Executor</span><span class="w"> </span><span class="n">executor</span><span class="p">;</span>

<span class="w">  </span><span class="k">const</span><span class="w"> </span><span class="kt">size_t</span><span class="w"> </span><span class="n">num_lines</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">4</span><span class="p">;</span>
<span class="w">  </span>
<span class="w">  </span><span class="c1">// create a pipeline graph</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">DataPipeline</span><span class="w"> </span><span class="n">pl</span><span class="p">(</span><span class="n">num_lines</span><span class="p">,</span>
<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">make_data_pipe</span><span class="o">&lt;</span><span class="kt">void</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="o">&gt;</span><span class="p">(</span><span class="n">tf</span><span class="o">::</span><span class="n">PipeType</span><span class="o">::</span><span class="n">SERIAL</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="o">&amp;</span><span class="p">](</span><span class="n">tf</span><span class="o">::</span><span class="n">Pipeflow</span><span class="o">&amp;</span><span class="w"> </span><span class="n">pf</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="kt">int</span><span class="p">{</span>
<span class="w">      </span><span class="k">if</span><span class="p">(</span><span class="n">pf</span><span class="p">.</span><span class="n">token</span><span class="p">()</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="mi">5</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">        </span><span class="n">pf</span><span class="p">.</span><span class="n">stop</span><span class="p">();</span>
<span class="w">        </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="w">      </span><span class="p">}</span>
<span class="w">      </span><span class="k">else</span><span class="w"> </span><span class="p">{</span>
<span class="w">        </span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;first pipe returns %lu</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span><span class="w"> </span><span class="n">pf</span><span class="p">.</span><span class="n">token</span><span class="p">());</span>
<span class="w">        </span><span class="k">return</span><span class="w"> </span><span class="n">pf</span><span class="p">.</span><span class="n">token</span><span class="p">();</span>
<span class="w">      </span><span class="p">}</span>
<span class="w">    </span><span class="p">}),</span>

<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">make_data_pipe</span><span class="o">&lt;</span><span class="kt">int</span><span class="p">,</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&gt;</span><span class="p">(</span><span class="n">tf</span><span class="o">::</span><span class="n">PipeType</span><span class="o">::</span><span class="n">SERIAL</span><span class="p">,</span><span class="w"> </span><span class="p">[](</span><span class="kt">int</span><span class="o">&amp;</span><span class="w"> </span><span class="n">input</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">      </span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;second pipe returns a strong of %d</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span><span class="w"> </span><span class="n">input</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">100</span><span class="p">);</span>
<span class="w">      </span><span class="k">return</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">to_string</span><span class="p">(</span><span class="n">input</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">100</span><span class="p">);</span>
<span class="w">    </span><span class="p">}),</span>

<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">make_data_pipe</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span><span class="w"> </span><span class="kt">void</span><span class="o">&gt;</span><span class="p">(</span><span class="n">tf</span><span class="o">::</span><span class="n">PipeType</span><span class="o">::</span><span class="n">SERIAL</span><span class="p">,</span><span class="w"> </span><span class="p">[](</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&amp;</span><span class="w"> </span><span class="n">input</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">      </span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;third pipe receives the input string %s</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span><span class="w"> </span><span class="n">input</span><span class="p">.</span><span class="n">c_str</span><span class="p">());</span>
<span class="w">    </span><span class="p">})</span>
<span class="w">  </span><span class="p">);</span>

<span class="w">  </span><span class="c1">// build the pipeline graph using composition</span>
<span class="w">  </span><span class="n">taskflow</span><span class="p">.</span><span class="n">composed_of</span><span class="p">(</span><span class="n">pl</span><span class="p">).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;pipeline&quot;</span><span class="p">);</span>

<span class="w">  </span><span class="c1">// dump the pipeline graph structure (with composition)</span>
<span class="w">  </span><span class="n">taskflow</span><span class="p">.</span><span class="n">dump</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">cout</span><span class="p">);</span>

<span class="w">  </span><span class="c1">// run the pipeline</span>
<span class="w">  </span><span class="n">executor</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">taskflow</span><span class="p">).</span><span class="n">wait</span><span class="p">();</span>

<span class="w">  </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="p">}</span></pre><p>The interface of <a href="classtf_1_1DataPipeline.html" class="m-doc">tf::<wbr />DataPipeline</a> is very similar to <a href="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr />Pipeline</a>, except that the library transparently manages the dataflow between pipes. To create a stage in a data-parallel pipeline, you should always use the helper function <a href="namespacetf.html#a8975fa5762088789adb0b60f38208309" class="m-doc">tf::<wbr />make_data_pipe</a>:</p><pre class="m-code"><span class="n">tf</span><span class="o">::</span><span class="n">make_data_pipe</span><span class="o">&lt;</span><span class="kt">int</span><span class="p">,</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&gt;</span><span class="p">(</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">PipeType</span><span class="o">::</span><span class="n">SERIAL</span><span class="p">,</span><span class="w"> </span>
<span class="w">  </span><span class="p">[](</span><span class="kt">int</span><span class="o">&amp;</span><span class="w"> </span><span class="n">input</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"> </span>
<span class="w">    </span><span class="k">return</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">to_string</span><span class="p">(</span><span class="n">input</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">100</span><span class="p">);</span>
<span class="w">  </span><span class="p">}</span>
<span class="p">);</span></pre><p>The helper function starts with a pair of an input and an output types in its template arguments. Both types will always be decayed to their original form using <a href="http://en.cppreference.com/w/cpp/types/decay.html" class="m-doc-external">std::<wbr />decay</a> (e.g., <code>const int&amp;</code> becomes <code>int</code>) for storage purpose. In terms of function arguments, the first argument specifies the direction of this data pipe, which can be either <a href="namespacetf.html#abb7a11e41fd457f69e7ff45d4c769564a7b804a28d6154ab8007287532037f1d0" class="m-doc">tf::<wbr />PipeType::<wbr />SERIAL</a> or <a href="namespacetf.html#abb7a11e41fd457f69e7ff45d4c769564adf13a99b035d6f0bce4f44ab18eec8eb" class="m-doc">tf::<wbr />PipeType::<wbr />PARALLEL</a>, and the second argument is a callable to invoke by the pipeline scheduler. The callable must take the input data type in its first argument and returns a value of the output data type. Additionally, the callable can take a <a href="classtf_1_1Pipeflow.html" class="m-doc">tf::<wbr />Pipeflow</a> reference in its second argument which allows you to query the runtime information of a stage task, such as its line number and token number.</p><pre class="m-code"><span class="n">tf</span><span class="o">::</span><span class="n">make_data_pipe</span><span class="o">&lt;</span><span class="kt">int</span><span class="p">,</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&gt;</span><span class="p">(</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">PipeType</span><span class="o">::</span><span class="n">SERIAL</span><span class="p">,</span><span class="w"> </span>
<span class="w">  </span><span class="p">[](</span><span class="kt">int</span><span class="o">&amp;</span><span class="w"> </span><span class="n">input</span><span class="p">,</span><span class="w"> </span><span class="n">tf</span><span class="o">::</span><span class="n">Pipeflow</span><span class="o">&amp;</span><span class="w"> </span><span class="n">pf</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">    </span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;token=%lu, line=%lu</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span><span class="w"> </span><span class="n">pf</span><span class="p">.</span><span class="n">token</span><span class="p">(),</span><span class="w"> </span><span class="n">pf</span><span class="p">.</span><span class="n">line</span><span class="p">());</span>
<span class="w">    </span><span class="k">return</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">to_string</span><span class="p">(</span><span class="n">input</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">100</span><span class="p">);</span>
<span class="w">  </span><span class="p">}</span>
<span class="p">)</span></pre><aside class="m-note m-info"><h4>Note</h4><p>By default, <a href="classtf_1_1DataPipeline.html" class="m-doc">tf::<wbr />DataPipeline</a> passes the data in reference to your callable at which you can take it in copy or in reference depending on application needs.</p></aside><p>For the first pipe, the input type should always be <code>void</code> and the callable must take a <a href="classtf_1_1Pipeflow.html" class="m-doc">tf::<wbr />Pipeflow</a> reference in its argument. In this example, we will stop the pipeline when processing five tokens.</p><pre class="m-code"><span class="n">tf</span><span class="o">::</span><span class="n">make_data_pipe</span><span class="o">&lt;</span><span class="kt">void</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="o">&gt;</span><span class="p">(</span><span class="n">tf</span><span class="o">::</span><span class="n">PipeType</span><span class="o">::</span><span class="n">SERIAL</span><span class="p">,</span><span class="w"> </span><span class="p">[](</span><span class="n">tf</span><span class="o">::</span><span class="n">Pipeflow</span><span class="o">&amp;</span><span class="w"> </span><span class="n">pf</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="kt">int</span><span class="p">{</span>
<span class="w">  </span><span class="k">if</span><span class="p">(</span><span class="n">pf</span><span class="p">.</span><span class="n">token</span><span class="p">()</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="mi">5</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">    </span><span class="n">pf</span><span class="p">.</span><span class="n">stop</span><span class="p">();</span>
<span class="w">    </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w">    </span><span class="c1">// returns a dummy value</span>
<span class="w">  </span><span class="p">}</span>
<span class="w">  </span><span class="k">else</span><span class="w"> </span><span class="p">{</span>
<span class="w">    </span><span class="k">return</span><span class="w"> </span><span class="n">pf</span><span class="p">.</span><span class="n">token</span><span class="p">();</span>
<span class="w">  </span><span class="p">}</span>
<span class="p">}),</span></pre><p>Similarly, the output type of the last pipe should be <code>void</code> as no more data will go out of the final pipe.</p><pre class="m-code"><span class="n">tf</span><span class="o">::</span><span class="n">make_data_pipe</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span><span class="w"> </span><span class="kt">void</span><span class="o">&gt;</span><span class="p">(</span><span class="n">tf</span><span class="o">::</span><span class="n">PipeType</span><span class="o">::</span><span class="n">SERIAL</span><span class="p">,</span><span class="w"> </span><span class="p">[](</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&amp;</span><span class="w"> </span><span class="n">input</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">  </span><span class="n">std</span><span class="o">::</span><span class="n">cout</span><span class="w"> </span><span class="o">&lt;&lt;</span><span class="w"> </span><span class="n">input</span><span class="w"> </span><span class="o">&lt;&lt;</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">})</span></pre><p>Finally, you need to compose the pipeline graph by creating a module task (i.e., tf::Taskflow::compoased_of).</p><pre class="m-code"><span class="c1">// build the pipeline graph using composition</span>
<span class="n">taskflow</span><span class="p">.</span><span class="n">composed_of</span><span class="p">(</span><span class="n">pl</span><span class="p">).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;pipeline&quot;</span><span class="p">);</span>

<span class="c1">// dump the pipeline graph structure (with composition)</span>
<span class="n">taskflow</span><span class="p">.</span><span class="n">dump</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">cout</span><span class="p">);</span>

<span class="c1">// run the pipeline</span>
<span class="n">executor</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">taskflow</span><span class="p">).</span><span class="n">wait</span><span class="p">();</span></pre><div class="m-graph"><svg style="width: 40.600rem; height: 17.900rem;" viewBox="0.00 0.00 406.00 178.50">
<g transform="scale(1 1) rotate(0) translate(4 174.5)">
<title>Taskflow</title>
<g class="m-cluster">
<title>cluster_p0x7ffc47e53358</title>
<polygon points="8,-91.25 8,-162.5 96,-162.5 96,-91.25 8,-91.25"/>
<text text-anchor="middle" x="52" y="-149" font-family="Helvetica,sans-Serif" font-size="10.00">Taskflow</text>
</g>
<g class="m-cluster">
<title>cluster_p0x7ffc47e53220</title>
<polygon points="104,-8 104,-162.5 390,-162.5 390,-8 104,-8"/>
<text text-anchor="middle" x="247" y="-149" font-family="Helvetica,sans-Serif" font-size="10.00">m1</text>
</g>
<g class="m-node m-flat">
<title>p0x1878a88</title>
<polygon points="87.75,-135.25 20.25,-135.25 16.25,-131.25 16.25,-99.25 83.75,-99.25 87.75,-103.25 87.75,-135.25"/>
<polyline points="83.75,-131.25 16.25,-131.25"/>
<polyline points="83.75,-131.25 83.75,-99.25"/>
<polyline points="83.75,-131.25 87.75,-135.25"/>
<text text-anchor="middle" x="52" y="-113.38" font-family="Helvetica,sans-Serif" font-size="10.00">pipeline [m1]</text>
</g>
<g class="m-node">
<title>p0x1878600</title>
<polygon points="247,-135.25 216.04,-117.25 247,-99.25 277.96,-117.25 247,-135.25"/>
<text text-anchor="middle" x="247" y="-113.38" font-family="Helvetica,sans-Serif" font-size="10.00">cond</text>
</g>
<g class="m-node m-flat">
<title>p0x18786e8</title>
<polygon points="166,-52 112,-52 112,-48 108,-48 108,-44 112,-44 112,-24 108,-24 108,-20 112,-20 112,-16 166,-16 166,-52"/>
<polyline points="112,-48 116,-48 116,-44 112,-44"/>
<polyline points="112,-24 116,-24 116,-20 112,-20"/>
<text text-anchor="middle" x="139" y="-30.12" font-family="Helvetica,sans-Serif" font-size="10.00">rt&#45;0</text>
</g>
<g class="m-edge">
<title>p0x1878600&#45;&gt;p0x18786e8</title>
<path stroke-dasharray="5,2" d="M234.19,-106.61C218.71,-94.97 192.12,-74.96 170.97,-59.05"/>
<polygon points="173.29,-56.42 163.2,-53.2 169.09,-62.01 173.29,-56.42"/>
<text text-anchor="middle" x="201.62" y="-71.75" font-family="Helvetica,sans-Serif" font-size="10.00">0</text>
</g>
<g class="m-node m-flat">
<title>p0x18787d0</title>
<polygon points="238,-52 184,-52 184,-48 180,-48 180,-44 184,-44 184,-24 180,-24 180,-20 184,-20 184,-16 238,-16 238,-52"/>
<polyline points="184,-48 188,-48 188,-44 184,-44"/>
<polyline points="184,-24 188,-24 188,-20 184,-20"/>
<text text-anchor="middle" x="211" y="-30.12" font-family="Helvetica,sans-Serif" font-size="10.00">rt&#45;1</text>
</g>
<g class="m-edge">
<title>p0x1878600&#45;&gt;p0x18787d0</title>
<path stroke-dasharray="5,2" d="M240.89,-102.47C236.05,-91.54 229.13,-75.91 223.16,-62.45"/>
<polygon points="226.51,-61.36 219.26,-53.63 220.11,-64.19 226.51,-61.36"/>
<text text-anchor="middle" x="233.62" y="-71.75" font-family="Helvetica,sans-Serif" font-size="10.00">1</text>
</g>
<g class="m-node m-flat">
<title>p0x18788b8</title>
<polygon points="310,-52 256,-52 256,-48 252,-48 252,-44 256,-44 256,-24 252,-24 252,-20 256,-20 256,-16 310,-16 310,-52"/>
<polyline points="256,-48 260,-48 260,-44 256,-44"/>
<polyline points="256,-24 260,-24 260,-20 256,-20"/>
<text text-anchor="middle" x="283" y="-30.12" font-family="Helvetica,sans-Serif" font-size="10.00">rt&#45;2</text>
</g>
<g class="m-edge">
<title>p0x1878600&#45;&gt;p0x18788b8</title>
<path stroke-dasharray="5,2" d="M253.11,-102.47C257.95,-91.54 264.87,-75.91 270.84,-62.45"/>
<polygon points="273.89,-64.19 274.74,-53.63 267.49,-61.36 273.89,-64.19"/>
<text text-anchor="middle" x="269.62" y="-71.75" font-family="Helvetica,sans-Serif" font-size="10.00">2</text>
</g>
<g class="m-node m-flat">
<title>p0x18789a0</title>
<polygon points="382,-52 328,-52 328,-48 324,-48 324,-44 328,-44 328,-24 324,-24 324,-20 328,-20 328,-16 382,-16 382,-52"/>
<polyline points="328,-48 332,-48 332,-44 328,-44"/>
<polyline points="328,-24 332,-24 332,-20 328,-20"/>
<text text-anchor="middle" x="355" y="-30.12" font-family="Helvetica,sans-Serif" font-size="10.00">rt&#45;3</text>
</g>
<g class="m-edge">
<title>p0x1878600&#45;&gt;p0x18789a0</title>
<path stroke-dasharray="5,2" d="M259.81,-106.61C275.29,-94.97 301.88,-74.96 323.03,-59.05"/>
<polygon points="324.91,-62.01 330.8,-53.2 320.71,-56.42 324.91,-62.01"/>
<text text-anchor="middle" x="309.62" y="-71.75" font-family="Helvetica,sans-Serif" font-size="10.00">3</text>
</g>
</g>
</svg>
</div></section><section id="UnderstandInternalDataStorage"><h2><a href="#UnderstandInternalDataStorage">Understand Internal Data Storage</a></h2><p>By default, <a href="classtf_1_1DataPipeline.html" class="m-doc">tf::<wbr />DataPipeline</a> uses <a href="https://en.cppreference.com/w/cpp/utility/variant">std::<wbr />variant</a> to store a type-safe union of all input and output data types extracted from the given data pipes. To avoid false sharing, each line keeps a variant that is aligned with the cacheline size. When invoking a pipe callable, the input data is acquired in reference from the variant using <a href="https://en.cppreference.com/w/cpp/utility/variant/get">std::<wbr />get</a>. When returning from a pipe callable, the output data is stored back to the variant using assignment operator.</p></section><section id="DataParallelPipelineLearnMore"><h2><a href="#DataParallelPipelineLearnMore">Learn More about Taskflow Pipeline</a></h2><p>Visit the following pages to learn more about pipeline:</p><ol><li><a href="TaskParallelPipeline.html" class="m-doc">Task-parallel Pipeline</a></li><li><a href="TaskParallelScalablePipeline.html" class="m-doc">Task-parallel Scalable Pipeline</a></li><li><a href="TextProcessingPipeline.html" class="m-doc">Text Processing Pipeline</a></li><li><a href="GraphProcessingPipeline.html" class="m-doc">Graph Processing Pipeline</a></li><li><a href="TaskflowProcessingPipeline.html" class="m-doc">Taskflow Processing Pipeline</a></li></ol></section>
      </div>
    </div>
  </div>
</article></main>
<div class="m-doc-search" id="search">
  <a href="#!" onclick="return hideSearch()"></a>
  <div class="m-container">
    <div class="m-row">
      <div class="m-col-m-8 m-push-m-2">
        <div class="m-doc-search-header m-text m-small">
          <div><span class="m-label m-default">Tab</span> / <span class="m-label m-default">T</span> to search, <span class="m-label m-default">Esc</span> to close</div>
          <div id="search-symbolcount">&hellip;</div>
        </div>
        <div class="m-doc-search-content">
          <form>
            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
          </form>
          <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
          <div id="search-help" class="m-text m-dim m-text-center">
            <p class="m-noindent">Search for symbols, directories, files, pages or
            modules. You can omit any prefix from the symbol or file path; adding a
            <code>:</code> or <code>/</code> suffix lists all members of given symbol or
            directory.</p>
            <p class="m-noindent">Use <span class="m-label m-dim">&darr;</span>
            / <span class="m-label m-dim">&uarr;</span> to navigate through the list,
            <span class="m-label m-dim">Enter</span> to go.
            <span class="m-label m-dim">Tab</span> autocompletes common prefix, you can
            copy a link to the result using <span class="m-label m-dim">⌘</span>
            <span class="m-label m-dim">L</span> while <span class="m-label m-dim">⌘</span>
            <span class="m-label m-dim">M</span> produces a Markdown link.</p>
          </div>
          <div id="search-notfound" class="m-text m-warning m-text-center">Sorry, nothing was found.</div>
          <ul id="search-results"></ul>
        </div>
      </div>
    </div>
  </div>
</div>
<script src="search-v2.js"></script>
<script src="searchdata-v2.js" async="async"></script>
<footer><nav>
  <div class="m-container">
    <div class="m-row">
      <div class="m-col-l-10 m-push-l-1">
        <p>Taskflow handbook is part of the <a href="https://taskflow.github.io">Taskflow project</a>, copyright © <a href="https://tsung-wei-huang.github.io/">Dr. Tsung-Wei Huang</a>, 2018&ndash;2024.<br />Generated by <a href="https://doxygen.org/">Doxygen</a> 1.9.6 and <a href="https://mcss.mosra.cz/">m.css</a>.</p>
      </div>
    </div>
  </div>
</nav></footer>
</body>
</html>
